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Abstract 


The hype around artificial intelligence (AI) in radiology continues unabated, despite the fact that the exact role AI will play in future 


radiology practice remains undefined. Nevertheless, education of the radiologists of the future is ongoing and needs to account for the 


uncertainty of this new technology. Radiology residency training has evolved even before the recent advent of imaging AI. Yet radiology 


residents and fellows will likely one day experience the benefits of an Al-enabled clinical training. This will offer them a customized 


learning experience and the ability to analyze large quantities of data about their progress in residency, with substantially less manual 


effort than is currently required. Additionally, they will need to learn how to interact with AI tools in clinical practice, and more 


importantly, understand how to evaluate AI outputs in a critical fashion as yet another piece of information contributing to the 


interpretation of an imaging examination. Although the exact role AI will play in the future practice of radiology remains undefined, it 


will surely be integrated into the education of future radiologists. 
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INTRODUCTION 

A great deal of attention is being paid to artificial intelligence 
(AI) in radiology, and with it comes a substantial amount of 
uncertainty and hype. Exactly what role AI will play in the 
future practice of radiology remains undefined. As a result, 
there is also uncertainty about how it will affect radiology 
residency training. Nevertheless, there are many potential 
ways AI could be used to teach future radiologists. The 
recent attention paid to AI and its capacity to detect imaging 
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findings has measurably affected medical students’ percep- 
tion of the field [1-3]. Framing the discussion of how AI will 
affect the trainee experience in the context of how radiology 
residents and fellows are taught today may serve to reassure 
future radiology residents that they will have a meaningful 
career augmented by AI. 


CURRENT APPROACHES TO RADIOLOGY 
EDUCATION 

The ACGME has substantial oversight of radiology resident 
education and has a set of common requirements for all 
residency programs. One of the goals implicit in the re- 
quirements is to facilitate the development of resident 
knowledge, skills, and values required to take ownership of 
patient care [4]. ACGME requirements specific for 
radiology resident education, how residency programs may 
satisfy those requirements, as well as some educational 
trends in providing residents with educational resources, 
didactics, workday teaching, and feedback will be discussed. 


Educational Resources 

ACGME requires that all radiology residency programs have 
sufficient learning resources available to trainees [1]. This 
includes readily available access to specialty specific 
material in electronic or format that must be able to 
support the number of residents within the radiology 
residency program. Radiology residencies can achieve this 
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in several ways including purchasing literature, online 
resources, and providing access to university libraries. 

Learning materials are conventionally in the form of 
textbooks, though review articles, online self-assessment 
platforms, and educational videos have been growing in 
popularity and may be preferred by some residents [5]. Most 
residency programs provide textbook recommendations and 
chapter-specific assignments for each rotation. However, 
reading, rereading, and highlighting textbooks may be a less 
effective form of learning [6,7]. This could be improved 
when reading becomes a more active process, whereby 
learners self-quiz, pause to consolidate information, and 
contrast disease processes [8-10]. Educational videos may be 
more useful to some learners and are becoming increasingly 
available from both radiology societies and commercial 
entities. Videos under 15 min have been growing in 
popularity due to their limited scope, which may make 
the information feel more digestible. Review articles are 
preferred by some residents due to ease of online access 
[11]. As a result, some residency programs provide not 
only textbook-specific reading recommendations but also 
suggestions for resident-appropriate video content and 
article collections. 

Testing and retrieval-based strategies, in comparison 
with textbook reading, have been shown to be more effective 
in improving knowledge retention. Online question banks 
are growing in popularity both for learning and resident 
assessment [12]. Online platforms in comparison with 
textbooks and articles are also more dynamic. These 
learning platforms allow for the creation of curated case 
sets to meet the educational needs of a learner. Cases can 
be arranged by subspecialty, modality, or disease process 
depending on learner needs. Online platforms also 
interleave cases that users have previously answered 
incorrectly, increasing the likelihood that users address 
knowledge deficiencies [13]. 

Radiology residency programs are also required by the 
ACGME to have rotation-specific goals and objectives that 
are readily available to all residents [1]. These can be 
institution specific or adopted and modified from national 
societies. For instance, the American Society of Thoracic 
Radiology has published learning guidelines for residents 
rotating through cardiothoracic imaging. The guidelines 
are comprehensive and structured, providing programs 
that may not have robust subspecialty imaging sections 
guidance from experts in the field. 


Didactics 

Protected didactic time for residents in training is required 
by the ACGME. It is expected that residents participate in 
structured didactic activities, which are broadly defined and 


can include lectures, conferences, asynchronous learning, 
simulations, and case discussions [1]. Most residency 
programs provide a daily lecture series, with topics divided 
among radiology subspecialties. Lectures are often 45 to 
60 min and delivered as a slide presentation. This 
approach can be powerful when used effectively. However, 
presentations run the risk of being text-heavy, leading 
some lecturers to read directly from slides with little audi- 
ence participation. Attention of learners typically wanes 
when content is delivered in this fashion, and knowledge 
retention also tends to be poor compared with other 
learning methods [14,15]. 

Lectures can be redesigned to increase learner retention 
and understanding [16-19]. The flipped classroom assigns 
trainees a prelearning assignment before the didactic, such 
as an article or video. In class, participants have the 
chance to apply learned concepts from the prelearning 
under the instruction of the lecturer. This process allows 
for corrective feedback from an expert and the chance for 
learners to flush out concepts. It also gives them an 
opportunity to learn from and teach their peers. Applying 
the flipped classroom to medical students rotating through 
a radiology clerkship has shown promise. Although there 
were no baseline differences in test scores between 
students in the flipped classroom and traditional lectures, 
flipped learning was associated with greater task value, 
increased academic achievement, and more positive 
emotions [20]. 

Software has also been developed that may lead to more 
engaging lectures. Polling software can seamlessly integrate 
with slide presentations and provide a real-time, graphical 
representation of answer choices for questions posed to the 
audience. The RSNA Diagnosis Live application (https:// 
live.rsna.org/) allows speakers to upload images so that 
participants can answer questions on their personal devices 
as well as interact with radiology images by selecting the 
abnormality [21,22]. Commercial tools also exist for either 
computers or tablets that simultaneously allow the speaker 
and one or more participants to draw on the lecturer’s 
slides from separate devices [23]. This can be particularly 
helpful for case conferences, where residents can circle or 


outline abnormalities. 


Learning During the Workday 

ACGME requires that a sufficient number of faculty be 
present to instruct and supervise all residents [4]. These 
faculty members must be role models of professionalism, 
demonstrate strong interest in resident education, and 
administer and maintain an educational environment 
conducive to educating residents [4]. Learning from 
faculty during the workday is usually more experiential 
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and mainly occurs when residents review examinations 
under the guidance of an expert radiologist at a PACS 
reading station. Learning in this setting is influenced by 
two elements: the educational ability of the supervising 
radiologist in conjunction with resident learning 
preference and the cases that are available for 
interpretation. Pairing of residents with an attending 
radiologist may be arbitrary and dictated by work 
schedules rather than educational styles and personalities. 
Some attention is usually given to resident case mix and 
work volume, although resident case exposure is usually 
driven by  worklist assignments developed for 
predominately clinical rather than educational needs. 
ACGME does have a required minimum number of 
modality-specific cases to be logged for residents over the 
course of their training [24]. 

During a rotation, a resident may be instructed to focus 
on studies that ready them for call (pulmonary embolism, 
traumas, stroke alerts) or a modality that they have gradu- 
ated to (MRIs). Work volume expectations may also be set. 
However, both volume and case mix are determined by 
referring providers and the imaging center’s capacity. Sur- 
veillance oncologic imaging may represent the majority 
imaging during the workday, limiting residents’ exposure to 
other disease processes as well as the supervising radiologist’s 
ability to address potential educational deficiencies. Radi- 
ology worklists are also limited in their ability to parse out 
interesting or potentially educationally noteworthy studies 
that could foster discussion and resident assessment. As a 
result, residents may not encounter some diseases or imaging 
modalities frequently enough to feel proficient. 

Radiology residencies have tried to address this by 
creating teaching files arranged by modality, specialty, or 
disease process. Radiology case-based simulations have been 
developed to round out resident experiences and call pre- 
paredness. Vendor-neutral digital teaching files allow radi- 
ologists to upload anonymized imaging studies from PACS 
with minimal disruption to clinical workflow [25]. Cases 
can be labeled, grouped into decks, and authored to 
highlight relevant teaching points. Case-based simulations 
in preparation for call have been shown to at least create a 
subjective feeling of preparedness for trainees [26]. 


Resident Evaluation and Feedback 

The ACGME requires training programs to maintain a 
learning environment conducive to educating residents in 
each of the six core competencies: professionalism, patient 
care and procedural skills, medical knowledge, practice- 
based learning and improvement, interpersonal and 
communication skills, and systems-based practice [4]. These 
core competencies provide a framework in developing 
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physicians entrusted to enter autonomous practice. 
Residency-specific milestones were created to assess resi- 
dents’ trajectory during their training in each of these six 
competencies and are used during ACGME-required clinical 
competence committee’s semi-annual review of resident 
performance [27]. Milestone performance data for each 
program’s residents are an element used in the ACGME’s 
Next Accreditation System to assess if residents are 
progressing and identify which residency programs are 
flourishing or struggling [28]. Information from this 
national database can identify programs that are doing 
well to help establish best practices and provide aid to 
those having difficulty [29]. 

During their training, residents may progress through 
five levels of milestones for each core competency. These 
levels do not necessarily correspond to years of training, such 
that a third-year resident does not have to be a level 3, and a 
junior resident may be at a higher milestone compared with 
a senior. Residents could also regress in milestone levels. 
Level 4 is designated as a graduation goal but is not a 
requirement for graduation, which is at the purview of the 
residency program director. Level 5 is considered to be an 
expert resident who has surpassed expectations. 

Core competencies are further divided into sub- 
compentency categories. For instance, “Patient Care” has 
four subcompetency categories (reporting, clinical consul- 
tation, image interpretation, and competence in procedures) 
and systems-based practice has eight, each with five levels of 
milestones. Each milestone level has language specific for the 
subcompetency being assessed. For example, in level 1 of the 
subcompentency “Diagnostic Knowledge,” the resident 
“demonstrates knowledge of imaging anatomy,” whereas in 
level 5, the resident “proficiently integrates knowledge of 
anatomic and molecular imaging with pathophysiology to 
formulate a diagnosis at the expected level of a subspecial- 
ist.” An updated version, Milestones 2.0, took effect in 
July 2020. 


HOW Al MIGHT CHANGE THE RADIOLOGY 
TRAINEE’S EXPERIENCE 


Al-Enabled Radiology Education 
The future impact of AI in radiology is not limited to its role 
in findings detection or its potential impact on workflow 
efficiency [30]. AI also has the potential to take large 
quantities of data about resident education, performance, 
and progress through training and enable customization of 
the educational experience to the strengths and weaknesses 
of the individual trainee [31,32]. 

As newer educational methods permeate radiology (eg, 
flipped classrooms, virtual education), incorporation of AI 
into education delivery can offer new methods for residents 
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to learn clinical radiology. For example, as the concept of 
gamification becomes a part of radiology education [33], 
trainees can earn rewards in an online platform for 
completing activities, such as achieving ACGME 
milestones, interpreting certain numbers or types of 
examinations, or passing online educational modules. AI 
can be a useful adjunct in helping residents to track their 
progress through training, by automatically identifying 
activities that are completed or milestones that are 
achieved and logging these for each trainee. At present, 
the logging process tends to be entirely or almost entirely 
manual, and adds work for each individual resident to 
keep track of their completed activities [34,35]. 

In addition, many radiology residency programs provide 
feedback to residents on the discrepancies between their on- 
call interpretations and the final attending overread [36,37]. 
This feedback may take the form of a message back to the 
interpreting resident that is manually entered by the 
faculty member or a text tag inserted in the report that 
can be mined and sent to a summary dashboard. 
Regardless of the form, the information is intended to 
provide residents with the opportunity to review cases on 
which they missed or misinterpreted findings and provide 
a valuable learning opportunity. By analyzing radiology 
resident performance using AI, there is greater potential to 
provide personalized resident feedback on either the basis 
of daytime rotation performance or off-hours call perfor- 
mance. Discrepancies with respect to the final attending 
interpretation can be analyzed to identify trends that could 
be aligned with the types of errors that typically occur in 
reports [38]. In turn, these could be summarized across 
residents to provide a number of relevant cases for review. 
Residents have often expressed concern that the feedback 
they receive is graded inconsistently; for example, one 
radiologist’s major discrepancy may be anothers minor 
discrepancy [39]. AI can potentially standardize resident 
feedback by analyzing large volumes of historic 
discrepancy data and providing guidance to both trainees 
and faculty. 

In addition to analysis of on-call interpretations, AI built 
into the dictation system could be used to identify areas of 
uncertainty that residents encounter while interpreting ex- 
aminations during daytime rotations. Furthermore, this in- 
formation could be used to tailor the types of examinations 
that a resident interprets by incorporating it into the reading 
work list [40]. For example, a resident who needs more 
experience with interstitial lung disease cases could 
preferentially receive these chest CTs on a given workday. 
Similarly, on a musculoskeletal radiology rotation, this 
would even out the distribution of joint cases and spine 
cases, such that the former is not preferentially dictated by 
the fellows, and the latter by the residents. 
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Al-Powered Radiologist Decision Support 

Although a great deal of attention is focused on clinical 
decision support for physicians ordering imaging, consid- 
erably less attention is focused on radiologist decision sup- 
port at the time of diagnosis. This conventionally takes the 
form of guidance regarding follow-up recommendations for 
lesions identified during interpretation of images [41]; 
however, AI could supply diagnostic support for both 
practicing radiologists and radiology trainees taking 
independent call [42]. One example of this is the use of 
Bayesian networks to offer differential diagnoses based on 


a set of described findings [43]. 


Al-Enabled Assessment of Radiology Trainees 
and Radiologists 

In light of changes to the ABR licensure process for diplo- 
mates over the past decade, the potential role of AI in 
evaluating radiology trainees’ preparedness for independent 
call and independent practice should also be considered. 


One study in the literature correlates USMLE examination Q6 


scores with the likelihood for subsequent poor performance 
on the ABR examinations [44]. However, more real-time 
assessment of residents during their training would be 
extremely valuable for identifying areas for further 
improvement in advance of sitting for licensure examina- 
tions. In the future, one could envision adaptive examina- 
tion of radiology trainees that is driven by AI to not only 
adjust to the level of knowledge of the candidate, but also to 
re-evaluate areas that may have previously been remediated 
[12]. Similar approaches could be applied for maintenance 
of certification as is currently required by the ABR [45]. 


Teaching Radiology Residents About Al 

In addition to considering how AI will change the trainee 
experience, it is important to consider how, when, and to 
what extent we should teach radiology residents and fellows 
about AI itself. The literature suggests that at minimum, 
radiology trainees will need to know how to use AI in their 
clinical work to understand where it can play a role and how 
to interpret its output [46]. One survey of current radiology 
residents and faculty found that nearly 85% of respondents 
felt that an understanding of AI should be taught during 
residency, and 80% felt it was as important to learn as 
imaging physics [47]. Another survey of practicing 
radiologists showed that they did not feel like they knew 
enough about AI and also wanted to learn more, so they 
might also benefit from the same education provided to 
trainees [48]. The challenge in adding more required 
components to the radiology residency arises when 
deciding what to remove or shorten to make room in the 
curriculum for new knowledge. Radiology residents will 
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likely need to be exposed to Al-related education 
throughout residency, ideally in the form of dedicated ses- 
sions as well as integrated into clinical teaching, as is 
currently done with medical physics. 


Potential Adverse Effects of Al 

As with every new technology, there will be potential 
adverse effects of AI that need to be addressed proactively. 
One popular proposed use for AI is triaging cases with ab- 
normalities to the top of the radiologist’s worklist, or having 
AI label normal examinations as such and remove them 
from the worklist entirely [49,50]. Although there are 
obvious efficiency benefits associated with removing 
normal cases from the worklist [49,51], the impact on 
radiology training should not be ignored. Radiology 
residents learn how to identify abnormalities by first 
establishing a mental standard for the normal appearance 
of a particular organ when captured by a particular 
imaging modality. Diverting these cases away from 
residents early in their training will thwart the normal 
learning process that radiology residents experience and 
require this knowledge to either be imparted in a different 
way or to limit junior residents’ exposure to AI triage 
tools. Another option would be to first teach junior 
residents the traditional methods of image interpretation 
before exposing them to AI; however, the practicality of 
this approach would have to be balanced against the 
pervasiveness of AI in clinical practice. 

A new challenge in training future radiologists will be 
teaching residents and fellows to recognize AI errors [50- 
52]. At present, evaluating software postprocessing of 
reconstructed images is primarily limited to reviewing 
measurements; these are generally easily recognized with 
training. However, AI errors may be more subtle, less 
predictable, and less repeatable, given the nature of the 
technology [52]. Susceptibility to automation bias—the 
idea that the computer is more correct than the human by 
virtue of being a machine—is greater when the human 
practitioner is less confident, and it could have a stronger 
impact on less experienced radiologists and radiology 
trainees [53]. Future radiology trainees will be taught to 
be mindful of automation bias, just as they currently learn 
to be wary of satisfaction of search [54,55]. 

In conclusion, as with new imaging modalities, PACS, 
and structured reporting, AI will undoubtedly require 
changes to radiology trainee education. However, there are 
many opportunities to integrate AI into the existing path- 
ways for radiology residents and fellows. AI will be yet 
another noninterpretive skill that radiology residents will be 
required to know. Current and future radiology residents 
and fellows will need to know how to interpret AI outputs, 
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in addition to knowing how to interpret images. They will 
need to understand how AI modifies the clinical workflow 
and be sensitive to AI outputs that seem unusual or 
incongruent. Although exactly what role AI will play in the 
future practice of radiology remains undefined, it will surely 
be integrated into the education of future radiologists. 


TAKE-HOME POINTS 


= Current educational approaches for radiology trainees 
include recommended readings, live and recorded 
lectures, and teaching at the workstation. Newer 
techniques such as flipped classroom sessions and 
polling can be used to enhance learning and the 


learner experience. 


= AI has the potential to take large quantities of data 
about resident education, performance, and progress 
through training and enable customization of the 
educational experience to the strengths and weak- 
nesses of the individual trainee. 


m AI could supply diagnostic support for both practicing 
radiologists and radiology trainees taking independent 


call. 


= Current and future radiology residents and fellows will 
need to understand how data should be curated for AI 
development, how AI tools can be developed, and how 
to interpret AI outputs, in addition to knowing how 
to interpret images. 


m The radiologists of the future will need to understand 
how AI modifies the clinical workflow and be sensitive 
to AI outputs that seem unusual or incongruent. 
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